Model explainers use statistical
techniques to calculate feature importance. This enables you to quantify
the relative influence each feature in the training data
set has on label prediction. Explainers work by evaluating a test
data set of feature cases and the labels the model predicts for them. Global feature importance quantifies
the relative importance of each feature in the test data set as a whole. It provides a general comparison of
the extent to which each feature in the dataset influences prediction. For example, a binary classification model
to predict loan default risk might be trained from features such as loan amount,
income, marital status and age to predict a label of one for loans that are likely
to be repaid and zero for loans that have a significant risk of default and
therefore shouldn't be approved. An explainer might then use a sufficiently
representative test data set to produce the following global feature
importance values, income 0.98, loan amount, 0.67, age 0.54 and
marital status 0.32. It's clear from these values that in
respect to the overall predictions generated by the model for the test data
set income is the most important feature for predicting whether or not a borrower
will default on a loan followed by the loan amount, then age and
finally marital status. Local feature importance measures
the influence of each feature value for a specific individual prediction. For example, suppose Sam applies for
a loan which the machine learning model approves by predicting that Sam
won't default on the loan repayment. You could use an explainer to calculate
the local feature importance for Sam's application to determine which
factors influence the prediction. Because this is a classification model, each feature gets a local importance
value for each possible class, indicating the amount of support for
that class based on the feature value. Since this is a binary
classification model, there are only two possible
classes zero and one. Each features support for one class results in a correlatively
negative level of support for the other. In Sam's case, the overall support for
class zero is -1.4 and the support for
class one is correspondingly 1.4. So support for class one is higher than
for a class zero and the loan is approved. The most important feature for a prediction of class one is
loan amount followed by income. These are the opposite order from their
global feature importance values which indicate that income is the most important
factor for the data sample as a whole. There could be multiple reasons why local
importance for an individual prediction varies from global importance for
the overall data set. For example, Sam might have
a lower income than average, but the loan amount in this case
might be unusually small. For a multi class classification model,
a local importance values for each possible classes calculated for every feature with the total across
all classes always being zero. For example, a model might predict
the species of a penguin based on features like it's bill length,
bill width, flipper length and weight. Suppose there are three
species of penguin. So the model predicts one of three
class labels zero, one or two. For an individual prediction, the flipper
length feature might have local importance values of 0.5 for class zero, 0.3 for
class one and -0.8 for class two. This indicated that the flipper length
moderately supports a prediction for class zero slightly supports a prediction
of class one, and it strongly supports the prediction of this particular penguin,
not being part of class two. For a regression model,
there are no classes, so the local importance value simply indicate
the level of influence each feature has on the predicted scale or label.